Markov Decision Processes

Accelerated decomposition techniques for large discounted Markov decision processes

Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...

Utilizing Generalized Learning Automata for Finding Optimal Policies in MMDPs

Multi agent Markov decision processes (MMDPs), as the generalization of Markov decision processes to the multi agent case, have long been used for modeling multi agent system and are used as a suitable framework for Multi agent Reinforcement Learning. In this paper, a generalized learning automata based algorithm for finding optimal policies in MMDP is proposed. In the proposed algorithm, MMDP ...

The KTH Visit in Semi-Markov Processes

On $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov‎ ‎processes

‎In the present paper we investigate the $L_1$-weak ergodicity of‎ ‎nonhomogeneous continuous-time Markov processes with general state‎ ‎spaces‎. ‎We provide a necessary and sufficient condition for such‎ ‎processes to satisfy the $L_1$-weak ergodicity‎. ‎Moreover‎, ‎we apply‎ ‎the obtained results to establish $L_1$-weak ergodicity of quadratic‎ ‎stochastic processes‎.

From Colored Petri Nets to Markov Decision Processes

 The present work is motivated by the need to consider stochastic behavior when planning production mix in a manufacturing system. Because methods that are suitable for planning are not always appropriate for modeling the systems and viceversa, two methods are combined here: systems are modeled using colored Petri nets and then converted into Markov decision processes. A special type of Petri ...

Optimal Control of Markov Regenerative Processes

In the paper the integration of available results on SemiMarkov Decision Processes and on Markov Regenerative Processes is attempted, in order to de ne the mathematical framework for solving decision problems where the underlying structure state process is a Markov Regenerative Process, referred to as Markov Regenerative Decision Process. The essential question investigated here is which descri...